Types of forests

• Random forest
• Boosting
  • Gradient boosting
  • Extreme gradient boosting (XG Boost)
  • Adaptive boosting (Ada Boost)

Random forest

• Create samples of same size as original by drawing observations randomly with replacement from the original sample.
• Fit a tree to each randomly generated sample.
• Average the trees to predict.

Predictions

• In each tree, answer the yes/no questions to find the leaf for an observation
• The prediction for that tree is
  • the leaf mean for regression
  • the leaf class probabilities for classification
• Repeat for each tree in the forest and average predictions
• For classification, the class with the highest average probability is the prediction.

Example

• Random forest regression for ranks.
• Get roeq and mom12m as before, and define rnk.
• Could also try return or numerical class (0, 1, 2).

Define and fit model

from sklearn.ensemble import RandomForestRegressor

X = data[["roeq", "mom12m"]]
y = data["rnk"]

model = RandomForestRegressor(
  max_depth=4,
  random_state=0
)
model.fit(X,y)

Things to do

• How well does it fit? Calculate \(R^2\) with model.score(X,y).
• How important is each of the predictors? Calculate with model.feature_importances_.
• Make a prediction for a new observation with model.predict.
• Sort on predictions. Do portfolios of stocks with high predictions have high returns?